Deep generative priors for biomolecular 3D heterogeneous reconstruction from cryo-EM projections.

Cryo-electron microscopy has become a powerful tool to determine three-dimensional (3D) structures of rigid biological macromolecules from noisy micrographs with single-particle reconstruction. Recently, deep neural networks, e.g., CryoDRGN, have demonstrated conformational and compositional heterogeneity of complexes. However, the lack of ground-truth conformations poses a challenge to assess the performance of heterogeneity analysis methods. In this work, variational autoencoders (VAE) with three types of deep generative priors were learned for latent variable inference and heterogeneous 3D reconstruction via Bayesian inference. More specifically, VAEs with "Variational Mixture of Posteriors" priors (VampPrior-SPR), non-parametric exemplar-based priors (ExemplarPrior-SPR) and priors from latent score-based generative models (LSGM-SPR) were quantitatively compared with CryoDRGN. We built four simulated datasets composed of hypothetical continuous conformation or discrete states of the hERG K + channel. Empirical and quantitative comparisons of inferred latent representations were performed with affine-transformation-based metrics. These models with more informative priors gave better regularized, interpretable factorized latent representations with better conserved pairwise distances, less deformed latent distributions and lower within-cluster variances. They were also tested on experimental datasets to resolve compositional and conformational heterogeneity (50S ribosome assembly, cowpea chlorotic mottle virus, and pre-catalytic spliceosome) with comparable high resolution. Codes and data are available: https://github.com/benjamin3344/DGP-SPR.

Hand Grasp Classification in Egocentric Video After Cervical Spinal Cord Injury.

OBJECTIVE: The hand function of individuals with spinal cord injury (SCI) plays a crucial role in their independence and quality of life. Wearable cameras provide an opportunity to analyze hand function in non-clinical environments. Summarizing the video data and documenting dominant hand grasps and their usage frequency would allow clinicians to quickly and precisely analyze hand function. METHOD: We introduce a new hierarchical model to summarize the grasping strategies of individuals with SCI at home. The first level classifies hand-object interaction using hand-object contact estimation. We developed a new deep model in the second level by incorporating hand postures and hand-object contact points using contextual information. RESULTS: In the first hierarchical level, a mean of 86% ±1.0% was achieved among 17 participants. At the grasp classification level, the mean average accuracy was 66.2 ±12.9%. The grasp classifier's performance was highly dependent on the participants, with accuracy varying from 41% to 78%. The highest grasp classification accuracy was obtained for the model with smoothed grasp classification, using a ResNet50 backbone architecture for the contextual head and a temporal pose head. DISCUSSION: We introduce a novel algorithm that, for the first time, enables clinicians to analyze the quantity and type of hand movements in individuals with spinal cord injury at home. The algorithm can find applications in other research fields, including robotics, and most neurological diseases that affect hand function, notably, stroke and Parkinson's.

Grasp Analysis in the Home Environment as a Measure of Hand Function After Cervical Spinal Cord Injury.

BACKGROUND: Following a spinal cord injury, regaining hand function is a top priority. Current hand assessments are conducted in clinics, which may not fully represent real-world hand function. Grasp strategies used in the home environment are an important consideration when examining the impact of rehabilitation interventions. OBJECTIVE: The main objective of this study is to investigate the relationship between grasp use at home and clinical scores. METHOD: We used a previously collected dataset in which 21 individuals with spinal cord injuries (SCI) recorded egocentric video while performing activities of daily living in their homes. We manually annotated 4432 hand-object interactions into power, precision, intermediate, and non-prehensile grasps. We examined the distributions of grasp types used and their relationships with clinical assessments. RESULTS: Moderate to strong correlations were obtained between reliance on power grasp and the Spinal Cord Independence Measure III (SCIM; P < .05), the upper extremity motor score (UEMS; P < .01), and the Graded Redefined Assessment of Strength Sensibility and Prehension (GRASSP) Prehension (P < .01) and Strength (P < .01). Negative correlations were observed between the proportion of non-prehensile grasping and SCIM (P < .05), UEMS (P < .05), and GRASSP Prehension (P < .01) and Strength (P < .01). CONCLUSION: The types of grasp types used in naturalistic activities at home are related to upper limb impairment after cervical SCI. This study provides the first direct demonstration of the importance of hand grasp analysis in the home environment.

3DFlex: determining structure and motion of flexible proteins from cryo-EM.

Modeling flexible macromolecules is one of the foremost challenges in single-particle cryogenic-electron microscopy (cryo-EM), with the potential to illuminate fundamental questions in structural biology. We introduce Three-Dimensional Flexible Refinement (3DFlex), a motion-based neural network model for continuous molecular heterogeneity for cryo-EM data. 3DFlex exploits knowledge that conformational variability of a protein is often the result of physical processes that transport density over space and tend to preserve local geometry. From two-dimensional image data, 3DFlex enables the determination of high-resolution 3D density, and provides an explicit model of a flexible protein's motion over its conformational landscape. Experimentally, for large molecular machines (tri-snRNP spliceosome complex, translocating ribosome) and small flexible proteins (TRPV1 ion channel, αVß8 integrin, SARS-CoV-2 spike), 3DFlex learns nonrigid molecular motions while resolving details of moving secondary structure elements. 3DFlex can improve 3D density resolution beyond the limits of existing methods because particle images contribute coherent signal over the conformational landscape.

Image Super-Resolution via Iterative Refinement.

We present SR3, an approach to image Super-Resolution via Repeated Refinement. SR3 adapts denoising diffusion probabilistic models (Ho et al. 2020), (Sohl-Dickstein et al. 2015) to image-to-image translation, and performs super-resolution through a stochastic iterative denoising process. Output images are initialized with pure Gaussian noise and iteratively refined using a U-Net architecture that is trained on denoising at various noise levels, conditioned on a low-resolution input image. SR3 exhibits strong performance on super-resolution tasks at different magnification factors, on faces and natural images. We conduct human evaluation on a standard 8× face super-resolution task on CelebA-HQ for which SR3 achieves a fool rate close to 50%, suggesting photo-realistic outputs, while GAN baselines do not exceed a fool rate of 34%. We evaluate SR3 on a 4× super-resolution task on ImageNet, where SR3 outperforms baselines in human evaluation and classification accuracy of a ResNet-50 classifier trained on high-resolution images. We further show the effectiveness of SR3 in cascaded image generation, where a generative model is chained with super-resolution models to synthesize high-resolution images with competitive FID scores on the class-conditional 256×256 ImageNet generation challenge.

3D variability analysis: Resolving continuous flexibility and discrete heterogeneity from single particle cryo-EM.

Single particle cryo-EM excels in determining static structures of protein molecules, but existing 3D reconstruction methods have been ineffective in modelling flexible proteins. We introduce 3D variability analysis (3DVA), an algorithm that fits a linear subspace model of conformational change to cryo-EM data at high resolution. 3DVA enables the resolution and visualization of detailed molecular motions of both large and small proteins, revealing new biological insight from single particle cryo-EM data. Experimental results demonstrate the ability of 3DVA to resolve multiple flexible motions of α-helices in the sub-50 kDa transmembrane domain of a GPCR complex, bending modes of a sodium ion channel, five types of symmetric and symmetry-breaking flexibility in a proteasome, large motions in a spliceosome complex, and discrete conformational states of a ribosome assembly. 3DVA is implemented in the cryoSPARC software package.

Non-uniform refinement: adaptive regularization improves single-particle cryo-EM reconstruction.

Cryogenic electron microscopy (cryo-EM) is widely used to study biological macromolecules that comprise regions with disorder, flexibility or partial occupancy. For example, membrane proteins are often kept in solution with detergent micelles and lipid nanodiscs that are locally disordered. Such spatial variability negatively impacts computational three-dimensional (3D) reconstruction with existing iterative refinement algorithms that assume rigidity. We introduce non-uniform refinement, an algorithm based on cross-validation optimization, which automatically regularizes 3D density maps during refinement to account for spatial variability. Unlike common shift-invariant regularizers, non-uniform refinement systematically removes noise from disordered regions, while retaining signal useful for aligning particle images, yielding dramatically improved resolution and 3D map quality in many cases. We obtain high-resolution reconstructions for multiple membrane proteins as small as 100 kDa, demonstrating increased effectiveness of cryo-EM for this class of targets critical in structural biology and drug discovery. Non-uniform refinement is implemented in the cryoSPARC software package.

cryoSPARC: algorithms for rapid unsupervised cryo-EM structure determination.

Single-particle electron cryomicroscopy (cryo-EM) is a powerful method for determining the structures of biological macromolecules. With automated microscopes, cryo-EM data can often be obtained in a few days. However, processing cryo-EM image data to reveal heterogeneity in the protein structure and to refine 3D maps to high resolution frequently becomes a severe bottleneck, requiring expert intervention, prior structural knowledge, and weeks of calculations on expensive computer clusters. Here we show that stochastic gradient descent (SGD) and branch-and-bound maximum likelihood optimization algorithms permit the major steps in cryo-EM structure determination to be performed in hours or minutes on an inexpensive desktop computer. Furthermore, SGD with Bayesian marginalization allows ab initio 3D classification, enabling automated analysis and discovery of unexpected structures without bias from a reference map. These algorithms are combined in a user-friendly computer program named cryoSPARC (http://www.cryosparc.com).

Building Proteins in a Day: Efficient 3D Molecular Structure Estimation with Electron Cryomicroscopy.

Discovering the 3D atomic-resolution structure of molecules such as proteins and viruses is one of the foremost research problems in biology and medicine. Electron Cryomicroscopy (cryo-EM) is a promising vision-based technique for structure estimation which attempts to reconstruct 3D atomic structures from a large set of 2D transmission electron microscope images. This paper presents a new Bayesian framework for cryo-EM structure estimation that builds on modern stochastic optimization techniques to allow one to scale to very large datasets. We also introduce a novel Monte-Carlo technique that reduces the cost of evaluating the objective function during optimization by over five orders of magnitude. The net result is an approach capable of estimating 3D molecular structure from large-scale datasets in about a day on a single CPU workstation.

Fast Exact Search in Hamming Space With Multi-Index Hashing.

There is growing interest in representing image data and feature descriptors using compact binary codes for fast near neighbor search. Although binary codes are motivated by their use as direct indices (addresses) into a hash table, codes longer than 32 bits are not being used as such, as it was thought to be ineffective. We introduce a rigorous way to build multiple hash tables on binary code substrings that enables exact k-nearest neighbor search in Hamming space. The approach is storage efficient and straight-forward to implement. Theoretical analysis shows that the algorithm exhibits sub-linear run-time behavior for uniformly distributed codes. Empirical results show dramatic speedups over a linear scan baseline for datasets of up to one billion codes of 64, 128, or 256 bits.

Shared Kernel Information Embedding for discriminative inference.

Latent variable models, such as the GPLVM and related methods, help mitigate overfitting when learning from small or moderately sized training sets. Nevertheless, existing methods suffer from several problems: 1) complexity, 2) the lack of explicit mappings to and from the latent space, 3) an inability to cope with multimodality, and 4) the lack of a well-defined density over the latent space. We propose an LVM called the Kernel Information Embedding (KIE) that defines a coherent joint density over the input and a learned latent space. Learning is quadratic, and it works well on small data sets. We also introduce a generalization, the shared KIE (sKIE), that allows us to model multiple input spaces (e.g., image features and poses) using a single, shared latent representation. KIE and sKIE permit missing data during inference and partially labeled data during learning. We show that with data sets too large to learn a coherent global model, one can use the sKIE to learn local online models. We use sKIE for human pose inference.

Model-Based 3D Hand Pose Estimation from Monocular Video.

A novel model-based approach to 3D hand tracking from monocular video is presented. The 3D hand pose, the hand texture, and the illuminant are dynamically estimated through minimization of an objective function. Derived from an inverse problem formulation, the objective function enables explicit use of temporal texture continuity and shading information while handling important self-occlusions and time-varying illumination. The minimization is done efficiently using a quasi-Newton method, for which we provide a rigorous derivation of the objective function gradient. Particular attention is given to terms related to the change of visibility near self-occlusion boundaries that are neglected in existing formulations. To this end, we introduce new occlusion forces and show that using all gradient terms greatly improves the performance of the method. Qualitative and quantitative experimental results demonstrate the potential of the approach.

TurboPixels: fast superpixels using geometric flows.

We describe a geometric-flow-based algorithm for computing a dense oversegmentation of an image, often referred to as superpixels. It produces segments that, on one hand, respect local image boundaries, while, on the other hand, limiting undersegmentation through a compactness constraint. It is very fast, with complexity that is approximately linear in image size, and can be applied to megapixel sized images with high superpixel densities in a matter of minutes. We show qualitative demonstrations of high-quality results on several complex images. The Berkeley database is used to quantitatively compare its performance to a number of oversegmentation algorithms, showing that it yields less undersegmentation than algorithms that lack a compactness constraint while offering a significant speedup over N-cuts, which does enforce compactness.

Gaussian process dynamical models for human motion.

We introduce Gaussian process dynamical models (GPDM) for nonlinear time series analysis, with applications to learning models of human pose and motion from high-dimensionalmotion capture data. A GPDM is a latent variable model. It comprises a low-dimensional latent space with associated dynamics, and a map from the latent space to an observation space. We marginalize out the model parameters in closed-form, using Gaussian process priors for both the dynamics and the observation mappings. This results in a non-parametric model for dynamical systems that accounts for uncertainty in the model. We demonstrate the approach, and compare four learning algorithms on human motion capture data in which each pose is 50-dimensional. Despite the use of small data sets, the GPDM learns an effective representation of the nonlinear dynamics in these spaces.

Synaptic vesicle mobility and presynaptic F-actin are disrupted in a N-ethylmaleimide-sensitive factor allele of Drosophila.

N-ethylmaleimide sensitive factor (NSF) can dissociate the soluble NSF attachment receptor (SNARE) complex, but NSF also participates in other intracellular trafficking functions by virtue of SNARE-independent activity. Drosophila that express a neural transgene encoding a dominant-negative form of NSF2 show an 80% reduction in the size of releasable synaptic vesicle pool, but no change in the number of vesicles in nerve terminal boutons. Here we tested the hypothesis that vesicles in the NSF2 mutant terminal are less mobile. Using a combination of genetics, pharmacology, and imaging we find a substantial reduction in vesicle mobility within the nerve terminal boutons of Drosophila NSF2 mutant larvae. Subsequent analysis revealed a decrease of filamentous actin in both NSF2 dominant-negative and loss-of-function mutants. Lastly, actin-filament disrupting drugs also decrease vesicle movement. We conclude that a factor contributing to the NSF mutant phenotype is a reduction in vesicle mobility, which is associated with decreased presynaptic F-actin. Our data are consistent with a model in which actin filaments promote vesicle mobility and suggest that NSF participates in establishing or maintaining this population of actin.